The Diamond Eye Image Mining System

Diamond Eye is a new image mining system that enables users (scientists) to locate and catalog objects of interest in large image collections. This system provides a platform-independent interface to novel image mining algorithms, as well as computational and database resources that allow scientists to browse, annotate, and search through images and analyze the resulting object catalogs. The Diamond Eye system is a distributed software architecture that employs a Java client (applet) frontend communicating with one or more custom Java servers. Each server uses an object-oriented database to provide persistent storage of structures used in the mining process and to enable queries over the mined information. Servers also make use of a computational engine, such as a network of high-performance workstations, to provide parallel execution of expensive image processing, object recognition, and query-by-content operations. Diamond Eye provides a number of essential image mining capabilities. At its core are novel image processing and object recognition algorithms that support automated search and retrieval of image content. The client interface includes tools for forming queries that can be used to search for speci c objects within a set of images. For example, the user can hand-label objects of interest in a small set of images and the system can then learn an object recognition model from the labeled examples. The system also provides tools for analyzing the query results. Object markers generated by an algorithm can be compared with human-generated labels to measure performance and provide feedback to the algorithm developer. Interactive browsers allow detected objects to be displayed as a mosaic of thumbnails, each with a link back to its source image. An object-oriented database associated with the server provides persistent storage of client data, query models, and results. The database schema includes a rich collection of classes to support the mining process: images, annotation markers, object recognition models, labeling events, algorithms, and users. The Image class can be easily subclassed to add domain speci c meta-data elements as required, including graphical associations between images. This allows a developer to organize the meta-data associated with a given image collection and enables the identi cation of subsets of images based on meta-data queries. Other classes support associations between images and the mined content and enable knowledge (e.g., useful query models and results) to be accumulated in the system. A prototype of Diamond Eye including persistent classes has been designed and implemented. Proof of concept examples for some database capabilities have been demonstrated using the ObjectStore object database management system. Persistent class and user interface design is continuing, with an emphasis on supporting storage and retrieval of recognition models and image search results. Our experiences with the Diamond Eye system show that it o ers numerous advantages over previous image mining systems. The software infrastructure is applicable to a wide range of science mining applications. Also, easy-access through a web-browser promotes trial evaluation of the system by potential new users. Finally, the system facilitates closer collaborations between image mining algorithm developers and domain experts, an essential part of the KDD process. [1] M.C. Burl, C. Fowlkes, J. Roden, A. Stechert, and S. Mukhtar, \Diamond Eye: A distributed architecture for image data mining", SPIE Aerosense'99, Data Mining and Knowledge Discovery: Theory, Tools, and Techniques, Orlando, FL, (April 1999) [2] M.C. Burl, C. Fowlkes, J. Roden, \Mining for Image Content", Proc. of Joint Conf. on Systemics, Cybernetics, and Informatics and Intelligent Systems: Analysis and Synthesis, (July 1999 to appear)